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I. INTRODUCTION 



During September and early October 1970, the development status of 
the Center for Information Services project was carefully reviewed by 
representatives of the Institute of Library Research (ILR) , the Campus 
Computing Network (CCN) , and the University Library. While Phase IIA 
work continued , planning activities for Phase HR were initiated by 
ILR to provide a basis for continued CIS development and to provide a 
framework for the Phase IIB proposal document. 

In view of the requirement to plan for the integration of CIS into 
the University Library environments first at UCLA and eventually at the 
other UC campuses to meet CIS/LSD (Library Systems Development project 
of the University of California) integration needs, several areas were 
identified as needing study and clarification to support the continuing 
phases of CIS development. Included were items such as: task identifi- 

cation, scheduling, documentation guidance, user needs, data base 
availability, operational procedures, university support, system test 
and evaluation, and development responsibility. 



SCOPE 

This portion (Part 4) of the Phase IIA Final Report contains 
development schedules, task assignments, and discussions of pertinent 
Issues as they relate to Phase IIB. The information concerning schedules, 
assignments, and priorities reflects the currently funded 18 month 
development period for Phase IIB (1/71-6/72) rather than the 12 month 
period proposed in the Phase IIB supplement to the original Phase II 
Proposal, During this phase (Phase IIB) and Phase III, prime development 
responsibility lies with CCN, At the end of Phase III, prime 
responsibility will transfer to the Library during implementation and 
will remain with the Library, After Phase IV, operational responsibility 
including full economic support lies with the University Library, 

During Phase IIB, the Institute of Library Research, Campus Computing 
Network, and the University Library will jointly participate in developing 
and providing mechanized information services to the University community, 
CCN is concerned with information processing system design and operation; 
the Library is oriented toward data base acquisition, procedures, and 
experimental services; and ILR activities include coordination, 
documentation, and considerations of system test and evaluation. The 
tasks, allocations , and schedules as presented will undoubtedly change 
over time to reflect CCN and Library ordering of priorities, available 
funds, and NSF-approved development schedules. 
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Sections XX, XXI, and IV (including Appendix A) contain current 
schedule, organization, and task information accompanied by the 
appropriate identification of responsibilities within UCLA and within 
the project. Section V identifies the several committees, including 
established User Committees, already formed to help guide CIS develop- 
ment , The final sections (VI through IX) provide resource information 
pertinent to the remaining development and operational phases of CIS. 
For those unfamiliar with this development, the following paragraphs 
review briefly the rationale and concept of a Center for Information 
Services, A more complete description will be found in Part 3 of the 
Phase IIA Final Report "Software System Design and Related Software 
Activities . ” 



CENTER FOR INFORMATION SERVICE S 

The purpose of the proposed Center for Information Services is to 
provide a location on the UCLA campus at which data bases becoming 
available from national resources can be acquired, stored, and made 
available to the research community of the University of California, 
Many of the issues to be resolved are technical but other equally 
important issues to be investigated and resolved are administrative, 
legal, jurisdictional, and economic in nature. Many aspects of CIS 
including the above issues were investigated and the results recorded 
during Phase I. Design goals and system development issues have been 
discussed in the basic Phase II proposal and Phase IIA and Phase IIB 
supplements ; many are repeated in this document. 

The operational Center for Information Services as presently 
conceived services the user community through one or more communication 
channels, the selection determined by the nature of the request and the 
appropriate response configuration. As shown on Figure 1, most of the 
user requests are channeled through information specialists. Service 
responses may come directly from the University Library, from the Campus 
Computing Network (CCN), or from both. 

In this model the Institute of Library Research (ILR) does not 
perform a direct service function but supports a program of research 
and evaluation that interacts with all elements of the system. The 
Library provides the necessary service functions except for the actual 
computer processing, which will be performed by CCN, In addition to 
initially developing and implementing the system design (Phase II), 

CCN will continue to provide the support required for system maintenance 
and improvement. The user community, viewed initially with the help of 
user committees, will represent a wide spectrum of needs, capabilities, 
and required services . 

Not all of the development issues are technical or such that they 
are readily solved by software development and hardware installation. 
Three examples of issues which must be resolved by the University and 
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OPERATIONAL DIAGRAM 
CENTER FOR INFORMATION SERVICES 





the University Library as well as the project, are discussed below to 
emphasize the range of concern. The first describes a new role, that 
of the Information Specialist. Where does he fit in the University 
organizational structure? The second example is concerned with the 
transfer of CIS operations to the Library. Will the funds be budgeted? 
Can costs be recovered? The final paragraph alludes to the interface 
of CIS development with total UC library automation efforts. How can 
they be integrated? 

The Information Specialist 

Since it is clear, at least initially, that most users of the CIS 
information processing system will not have the credentials or interest 
to use the computer programs directly, we propose to establish a group 
of people, called information specialists, who will act as an interface 
between the user community and the computer. In this capacity, they will 
not only control the data flow between the user and the computer, but 
will also have major responsibilities in such areas as user feedback and 
evaluation, user consulting, interfacing with the CCN programming staff, 
and preparing and maintaining documentation for users. In Phase I IB 
documentation will be prepared detailing the role of an information 
specialist; following is an outline of the areas to be detailed. 

Input and Output Handling . The information specialist will receive 
requests from users, and if the requests are not in a machine-readable 
format, he will prepare them in a manner appropriate for use by the 
computer. He will be knowledgeab3.e of the programs to be used, at least 
in an operational capacity. Similarly, he will collect output from the 
computer and distribute it to the users. Data bases (i.g,, magnetic 
tapes) will be given to him and he will then place them in the tape 
library at CCN, or, when necessary, pass them on the programming support 
staff if they are not in a useable state. 

User Feedback and Evaluation . The information specialist will 
collect and document users r usage feedback (from interviews, question- 
naires, etc.) to be used for three chief purposes: to identify new 

applications, detect operational errors, and identify areas in which 
CIS is failing users 1 needs. 

User Consulting . The information specialist will consult with 
users in order to assist them in two principal areas : (1) ways In; which 

CIS may be used to solve their information-processing problems, and (2) 
special problems that individual users may encounter (e.g„, phrasing of 
profiles, interpreting output, etc.). 

Programming Interface , Although the information specialist will 
not be intimately involved with programming details, he will report to 
the programming staff the results of his user evaluation studies so that 
necessary changes can be implemented. If he discovers that a user has 



received erroneous results, he will give this information to the pro- 
gramming staff so that system flaws can be corrected. The information 
specialist will also report system changes to the user community if 
those changes affect how users use the system. 

Documentation . The information specialist will prepare and main- 
tain documents for use by users-at-large, He will also prepare from 
time to time documents discussing usage and evaluation of the system, 
including special problems that may occasionally arise. 



Transfer of CIS Operation to the Library 

An important Phase IIB task will be the specifications of procedures 
for the transfer of the operation of CIS to the Library, Specifications 
to be developed include procedures for administration and operation, job 
descriptions for CIS staff, and the funding for CIS. 



The operational procedures will be developed by the Library with 
assistance from ILR and CCN„ The procedures include operation of the 
CIS software, acquisition, indexing, and storage of the data bases, and 
how and when the user obtains service. 



The staff required for the CIS operation must be identified and job 
descriptions written. The information specialist has already been 
identified and some thought given to his duties. During Phase IIB the 
information specialist’s task will be handled jointly by CCN and Library 
personnel. The experience gained in Phase IIB will help in producing the 
job description. 



Fund ing . The funding of an operational CIS must be studied in some 
detail. It has always been envisioned that CIS will be an addition to 
the Library, Cost-sharing among the users of CIS will be investigated. 

University Commitment . The Center for Information Services is to 
become a completely operational service which meets the daily neejs of 
the University community. The commitment of the University to this 
project and to its continuation as an operational service was stated in 
the Supplement to the Phase II proposal (23 January 1969) , and also in 
letters from Chancellor Charles E. Young to Robert M, Hayes (7 January 
1963) and from Vice Chancellor David Saxon to Edward C. Weiss (15 September 
. ,1970). •' V N ‘ 



his letted Chanc^lor Young confirms ’rthe commitment of the 
/ Canipus. Administration not only to the development of the ’Center for 
h" Information Servi ces ’ which you proposed, but to its continuing 

operation.^ “Regarding the continued, operational funding of the CIS 
; Sservice&i V states- in his letter that "three types 

€}|^of cpi^dan be identified^ acquisition,: services of personnel, and 

computer processing services. The first is likely to be, funded both by 




users ... and by special university budgets for acquisition of data bases 
of broad utility.,.. The second is likely to be budgeted as part of 
university supported staff in the library and the c omp u ting facility. 

The third is likely to be funded by direct charges to users for services, 
with intra-mural funds available where they are needed, " In summary, _ 

Vice Chancellor Saxon points out that "the Chancellor has reaffirmed hxs 
commitment to the project ,, .and to the need for finding appropriate means 
of meeting the operational costs following the end of NSF support, " 

In a letter from Robert Vosper to Robert Hayes (6 January 1969 ) , 

Mr. Vosper discusses the Library T s commitment to providing the services 
being developed by CIS, and his interest in resolving the fiscal problems 
of an operational CIS* 

On November 3, 1970 a meeting was held with Robert Vosper, William 
Kohl , and Adrian Harris, the Director of Planning for UCLA. At that ^ 
meeting Adrian Harris agreed that "UCLA will recognize the CIS need in 
putting together our campus budget proposal for 1973-74 (start up funds 
only) and in 1974-75. The amount of need will not be known until the 
project is at least partially completed. We will rely on the Library 
to provide input data for the budget proposal." 

Interaction With The Library Systems Development Group 

The University Library Systems Development (LSD) project involves 
the use of computers to automate many of the library functions. It is 
being developed under the joint effort of all nine campuses, each of 
which is participating in the program development, CIS is a UCLA project 
effort. The results of the CIS project will be made available to the 
University-wide Library Systems Development as appropriate. However, an 
arrangement whereby data base problems affecting all campuses could be 
investigated now for early resolution is highly desirable. Support by 
the University during these formative stages of CIS development would 
significantly shorten the time that will otherwise be required to resolve 
inter— campus and Intra-campus issues of compatibility, vested interests, 
legal restrictions, duplicative development and processing efforts, impact 
on communication requirements, on-line inquiry by remote (aampus-to-campus) 
terminal,, and the development of effective user charge policies. 



II. development schedule 



LONG RANGE PLANS 

, During the final quarter of Phase IIA much attention was given to 
the long range plans for CIS, Planning included the clarification of 
the phases of the project, the identification of the major tasks to be 
° rm ed each group during each phase, personnel requirements, costs 
and funding, and the Phase JIB task schedule. Discussions of these 
topics were included in the proposal submitted to NSF for Phase IXB but, 
after the Phase JIB proposal* was submitted, it was necessary to change 
the timing of Phase IIB from 1 January 1971 - 31 December 1971 to 
Jr,^? nuary 1971 “ 30 June 1972, and that of Phase III from 1 January 
1972 - 30 June 1973 to 1 July 1972 - 30 June 1973. The revised schedule 
and some of the implied constraints are reflected in the discussions 
which follow. Preliminary development plans based on the initial time 
frame are included in Appendix A. 



Clarification of Phases 



In the original proposal the developmental plan of CIS was divided 
into the following four phases: 



Phase I , 



and Preliminary Specifications 



Phase II. Detailed System and Programming Design 
Phase III.. Computer Programming Development and Test 



Phase I 



and Initial Operation 

/ 

Because of. limited funding, it was necessary to divide Phase II into 
two parts, A and B. Two supplements to the original Phase II proposal 
havemow been written (Phase IIA, Phase IIB) to reflect funding constraint 
and to extend the time period of phase II, To clarify any confusion that 
may have arisen because of the change, the phases as they now envisioned 
are described below. The primary difference is the inclusion of the de- 
velopment of the prototype system in Phase II. This change is possible 
because of the decision that the computer programming could be done oom- 
"in-house” at UCLA. 



* 



Phase I IB Supplement to A Proposal for Devel opment of a Center for 
Information Services . Campus Computing Network., "uHiyersZty of 
, Los Angeles, November 17, 1970. 
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Phase I; Feasibility and Preliminary Specifications, ('Completed) 
This phase has been completed and is described in a 13-part report to 
N8F (see "Mechanized Information Services in the University Library," 

NSF Grant GN-503, Institute of Library Research, UCLA, 15 December 1967). 

Phase IIA : Requirement Specification and Basic Design (1 July 1969 

to 31 December 1970) , ('Completed') This phase has been completed and 
this document is one of the set of 7 that comprises the Final Report, 
Briefly, this phase saw the organization of CIS user committees, organ- 
ization of a series of seminars on CIS for library personnel, experi- 
mentation with a variety of sample data bases, installation and initial 
operation of two interim software systems, identification of documents 
and procedures required for further CIS development, and the basic design 
of the CIS software system. 

Phase I1B; Detailed Design and Prototype Development (1 January 1971 
to 30 June 1972) . Transfer of project responsibility from ILR to CCN took 
place at the start of this phase when Mr, William Kehl, Director of CCN, 
became the Principal Investigator, Activities will center around 
experimental operations with interim software, additional seminars for 
library personnel, development of operational procedures for transfer of 
the operation to the Library, development of test and evaluation studies, 
acquisition of selected data bases, development of a prototype using 
procedures developed during this phase. Development of the prototype 
will be the major task of this phase. 

Phase III: Prototype Operation and Operational System Completion 

(1 July 1972 to 30 June 1973) , During Phase III the prototype system 
will be in operation with a selected set of services while the remainder 
of the software system, including interactive capabilities under the 
Time Sharing Option of the IBM Operating System 360, is completed. The 
prototype will be operated jointly by CCN and the Library, Procedures 
for the operational system will be completed and a phased transition of 
the operational responsibility to the Library will begin. Extensive test 
and evaluation of the total CIS operation will take place to assist the 
revision and refinement of policies and procedures* Acquisition of data 
bases will continue. 

Phase IV: Implementation and Evaluation of Services (1 July 1973 

to June 50 1974) , Formal transfer of the responsibility of the CIS 
project from CCN to the Library will take place at the beginning of this 
phase with Mr. Robert Vosper, University Librarian, becoming the Principal 
Investigator, The major task will be to make the transition from a 
developmental project to smooth- running operational service. Additional 
software services for users will be added in response to the evaluation 
study and to Users T experiences and requests. Operational policies and 
procedures will be improved as needed. Ongoing system evaluation 
procedures that will provide the information necessary to continually 
mold the system to fit the User community's needs will be implemented. 

The system software will he imp roved as needed to insure an efficient, 
cost-effective operation. - • 



MAJOR TASKS AND SUPPORT REQUIREMENTS 



The major tasks to be performed by each group are shown in Figure 2. 
For shared tasks, the group shown at the top of the bar is responsible 
for completion of the task. Note that the primary responsibility for the 
project shifts from ILR to CCN at the beginning of Phase IIB, and from 
CCN to the Library at the beginning of Phase IV. 

Personnel Requirements 

The estimated personnel requirements in full-time equivalents (FTE f s) 
are shown in Figure 3. These figures will probably be revised slightly 
as the project progresses. Although personnel for CCN and ILR are not 
shown for CIS after the first year of operation, assistance from both 
groups will be available if needed. Such needs may arise in the area of 
operational evaluation. 

Costs and Funding 

Estimated costs and funding for the remaining phases are shown in 
Figure 4, These estimates differ slightly from those given in the Phase 
IIB proposal in that computer time is included here in the CCN costs 
rather than in the Supplies and Expense costs. This is consistent with 
the notion that CIS is purchasing programming services from CCN. Note 
that these figures reflect the current estimates for the cost and funding 
areas given and most likely will change somewhat as the project progresses 
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PERSONNEL REQUIREMENTS 



FTE' s 










1 


PHASES 


LIBRARY 


CCN 


ILR 


ADMIN. 


TOTAL 


(by Quarter) 












IIB 1 


2 


9 


4 


.5 


15,5 


1/71 2 


2 


9 


4 


,5 


15.5 


to ? 


3 


9 


4 


.5 


16,5 


6/7 2 4 


4 


9 


3 


,5 


16.5 




4 


9 


3 


.5 


16.5 


6 


4 


9 


3 


.5 


16.5 


III 1 


5 


9 


3 


.5 


17.5 


7/72 2 


5 


9 


3 


.5 


17.5 


t n 


5 


9 


3 


, 5 


17.5 


, 4 

6/73 ' 


5 


8 


3 


.5 


16,5 


IV 1 


6 


7 


2,5 


,5 


16.0 


7/7 3 \ 


6 


5 


2.5 


,5 


14. 0 


to „ 


6 


3 


2.5 


.5 


12,0 


6/7 4 4 


6 


2 


1.5 


.5 


10.0 


1st Year 1 


6 


1 


0 


0 


7 .0 


7/7 4 | 


6 


1 


0 


0 


7.0 


to 4 

6/75 4 


6 

6 


1 

1 


0 

,0 


0 - 
0 


7.0 

7.0 


Subsequent 1 


7 


0 


0 


0 


7.0 


Years 2 


7 


0 


0 


0 


7,0 


. 3 


7 


0 


0 


0 


7.0 


4 


7 


' o 


0 


0 


7.0 
















FIGURE 4 



ESTIMATED COSTS AND FUNDING FOR CIS 
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III. ORGANIZATION 



PHASE I IB ADMINISTRATION 



■p responsibility was transferred from ILR to CCN at the start 

oi irtiase XIB. 

■ i 

Investigators 

.. As described In the previous section, William B. Kehl, Director of 
the Campus Computing Network, is designated Principal Investigator for 
Phase IIB. This is appropriate since a major effort during this phase 
is related to computer programming development. Robert Vesper UCLA 
University Librarian, will continue to serve as Co-Investigator. Start- 
ing with Phase IIB, Harold Berko, Professor in the Graduate School of 
Library Service and Robert L. Carmichael, Manager of the Institute of 
Library Research, Los Angeles, will become Co- Investigators. Professor 
Robert M. Hayes the Principal Investigator in the previous two phases 

of the project, has agreed to serve as an advisor to CIS during the 
coming year, e 

Task Management 

Specific tasks, defined in Section 3 of this document, are assigned 
to each of the three participating organizations. One person within each 
organization is responsible for the tasks assigned to that organization. 

In some instances, the people actually performing the work may belong to 
one of the other organizations f or the task team may have people repre- 
sent ing more than one organization. This arrangement worked well in 
Phase IIA and will be continued in Phase IIB, The keys to success are 
the high degree of communication and coordination maintained by these 
individuals and the joint dedication to the development of a Center for 
Information Services exhibited by all involved personnel. Project and 
organizational relationships are shown on Figure 5. “ 

Robert Carmichael, Manager of ILR, will be responsible for assigned 
ILR tasks involving project coordination , documentation control progress 
and cost reporting, and for initiating test and evaluation studies. (Funds 
tor the latter jtask have not been allocated in the current Phase IIB budget.) 
Professor Harold .Borko will coordinate the activities of the User Committees 
assist in establishing criteria and procedures for evaluation, and will 
contribute to the design of CIS from the point of view of the University 
Library and the Library School. ^ 
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FIGURE 5 

PROJECT ORGANIZATION 
UCLA CAMPUS 




Peter Watson, CIS Corrdinator for the Library, will be responsible 
for the Library 7 a tasks for Phase IIB. A librarian and two half-time 
research assistants ttfiXX assist him. The Library’s major tasks will be 
to acquire selected data bases, offer experimental services , develop 
operational procedures for CIS, and continue the seminar program. 

Bruce Briggs, CIS Project Manager for CCN, will be responsible for 
the completion of tasks assigned to CCN. He will be assisted by a staff 
of eight programmers. The major tasks of this group will be to complete 
the detailed design of the IPS, to develop and operate the prototype, and 
to provide interim experimental services. 

Cost and Progress Reporting 

The basic job cost system established during Phase IIA will be used 
to track and control costs during Phase IIB. Monthly cumulative cost 
summaries will be compared to the budget schedules prepared during the 
detailed planning for this phase. For this phase, acquisition costs are 
budgeted to provide selected data bases used to test experimental services 
and related library and computing procedures. These costs will be reported 
as they are incurred, hence the actual expenditure rate may vary around 
the predicted rate since establishing specific acquisition plans, require- 
ments and cost are Phase IXB tasks. Personnel budgeted for Phase IIB are 
shown on Figure 6, 

Three mechanisms will continue to be applied to monitor and report 
progress. Technical meetings will be held weekly, and policy meetings 
will be held monthly. Intra-CIS reports (working papers, cost reports, 
proposed procedures, etc.) will be prepared based on a schedule to be 
developed early in Phase IIB. General progress summaries will be 
submitted quarterly to the National Science Foundation. Technical working 
documents will be maintained in project files and will be used to prepare 
the Phase IIB final report. 



INSTITUTE OF LIBRARY RESEA RCJLRESPONSIBILIT IES 

During Phase IIB the Institute will be concerned primarily with 
administrative type responsibilities in support of CCN and Library efforts. 
Personnel limitations (see Figure 6) preclude broader support. Within 
this context, one function of the Institute of Library Research (ILR) 
during Phase IIB will be that of system coordination. It includes 
coordination of intra- CIS activities in order to insure that the tasks 
assigned to CCN and the UCLA Library are conducted in harmony and on 
schedule. It includes coordination of CIS documentation, particularly 
quarterly and final reports, ■ -i.. v 

Another function of ILR during Phase IIB will be to open and maintain 
lines of communication between CIS and the user community in order to 
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ORGANIZATION AND PERSONNEL 


NO. OF 
MONTHS 


NSF 

BUDGET 


UCLA 

CONTRIBUTION 


Canrous Computing Network 








W. Kehl, Prin. Investigator 


18 




10% 


B. Briggs 


18 


100% 


— 


A. de Boer 


18 


100% 


» 


P. Donahoe 


18 


100% 


— — 


W. Jordan 


18 


100% 




N. Kolb, Secretary 


18 


100% 




N. Ludlam 


18 


100% 




L. Miroff 


18 


100% 




J, Pine 


18 


50% 


» 


I, Riordan 


18 


50% 




University Library 








R. Vesper, Co. Prin. Iriv. 


18 


— 


10% 


P. Watson 


18 


100% 


— ■ 


(Librarian) 


12 


100% 




Research Assistant 


18 


50% 


— 


Research Assistant 


18 


50% 


— 


Institute of Library Research 








R. Carmichael, Co-Prin, Inv. 


18 


25% 




K (Secretary) 


18 


100% 


' — 


(Clerk-Typist) 


18 


50% 


V- 


Grad. School of Library Service 








H. Borko, Co-Prin. Inv. 


18 


1 ~ i\ 


xo% r/ 






familiarize the latter with CIS functions and to prepare the library 
staff in particular for the transition period in which they will be given 
control of CIS, This function will be accomplished by supporting seminars 
conducted for the Library staff, supporting user committees and user needs 
studies, and preparing brochures and newsletters explaining CIS activities 
In addition, ILR will continue to maintain the inventory of available data 
bases, which was compiled in Phase I of the project, and will continue 
interface discussions with the University of California Library Systems 
development Program, 



CAMPUS COMPUTING NETWORK RESPONSIBILITIES 

CCN has total responsibility for CIS development during Phase IIB, 

The capability to meet this requirement came about as follows. During 
Phase IIA, a component of the CIS project was established within the 
Campus Computing Network to develop the software needed by CIS, The 
primary task of this component was to develop a basic design of the 
Information Processing System for CIS. The status of these design ef- 
forts at the end of Phase IIA is contained in Part 3 of the Phase IIA 
Final Report. 

With respect to specific task allocations, during Phase IIB, CCN is 
responsible for competing the design and documenting the specifications 
and the programming details for the CIS Information Processing System, 

The primary product of this phase will be the development of the prototype 
system. In parallel with this design, documentation, and programming, CCN 
w il l conduct experimental operations with existing information retrieval 
systems in order to get operational experience that will influence the 
final design and the implementation of the CIS software, 

UNIVERSITY LIBRARY RESPONSIBILITIES 

The major responsibilities of the UCLA Library during Phase IIB are 
to work with user committees to identify desired data bases, acquire and 
maintain selected data bases, provide experimental services to the user 
community, develop viable operational procedures for the Center for 
Information Services, cooperate with ILR to conduct seminars for working 
librarians and interested users, work closely with the CCN and the ILR to 
prepare a general system specification, and prepare for the transfer of 
CIS operation to the Library. The Library will also work with CCN to 
provide needed information specialists and profile generation assistance. 




IV. PHASE I IB TASKS 



TASK SCHEDULE 

The major tasks and the scheduled task performance periods for Phase 
IIB are shown in Figure 7, For convenience of project organization, a 
task is listed under the group (organizational area) that has the major 
responsibility for that task. This organization of tasks, however, does 
not necessarily imply that a particular task will be accomplished totally 
by the group under which it is listed, but the list should provide a good 
guide to the distribution of activities and responsibilities. The com- 
parable task schedule as initially proposed for a 12 month rather than 
the current 18 month development period is shown in Appendix A where each 
task is briefly defined. Under the current schedule, the design activity 
plays a dominant role so as to fully utilize the experienced computer 
programmers who already are designing, coding, testing, and documenting 
elements of the prototype IPS. Several of the tasks areas are discussed 
in the following paragraphs to indicate key areas of Phase IIA activity 
and to record some of the objectives and plans for the current phase. 



IPS DESIGN AND DEVELOPMENT 
Requirements 

During Phase IIA, CON conducted a fairly detailed study of the 
requirements for the CIS Information Processing System (IPS). The 
results of this investigation are documented in Part 3 of the Phase IIA 
Final Report, and are summarized below to establish criteria for use In 
interpreting the nature of the tasks assigned to the CIS-CCN group. 

Services . The IPS must consist of a variety of services, including 
bibliographic, text-processing, and numerical services. Even within 
these services, there must be a considerable variety of subservices. For 
example, text-processing must include a number of services such as index- 
ing, concordance generation, frequency counts, and others. 

Data Formats . The IPS must be data-base independent. That is, it 
must not be structured around a few data base formats, but rather must be 
able to handle many existing formats as well as currently unforeseen ones 

Flexibility . The IPS must have a built-in flexibility to accept 
new services and functions. This is obvious since we cannot anticipate 
future applications of the system. We must be able to change or add 
components of the system without disturbing it. 
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Utilization, The IPS must be able to service many users simulta- 
neously. This requirement is motivated by two factors. First, for such 
operations as SDI work many users must be able to batch their profiles in 
order that only one pass of a tape is required rather than many. Second, 
users will want to explore data bases in an on-line mode. Hence there are 
two definitions of "simultaneous" in this context: many users using one 

service at the same time, and many users using many services at the same 
time. There is a need for servicing users both in a batch processing and 
an interactive mode. 

Instruction. The IPS should have a built-in instructional capability. 
This applies, of course, to an interactive mode. The instruction is 
intended mainly for beginning users who want to know how to use the IPS 
and the available data bases. Some terse instruction for more advanced 
users should be available as well. 

Recording, The IPS should have built-in accounting and monitoring 
functions for the purpose of billing, monitoring and evaluating system 
usage, and discovering bottlenecks to system efficiency. 

Usefulness . The IPS should be both efficient and cost effective. 
Within limits of practicality it should be exportable (i.e, usable at 
more than one location) . 



The major thrust of the GCN effort during Phase XIB is the development 
of the information processing system prototype, which will be expanded into 
the complete production version in Phases III and IV. The current goal is 
for GCN to have completed, operated, and tested, by the end of Phase IIB, 
a working prototype system that contains the following components: a moni- 
tor, a request manager, service managers, process managers, and a sample 
of translators, analyzers, and generators. An accounting and monitoring 
component will be included so that evaluation can begin with the 
implementation of the prototype. 

The implementation of on-line, interactive functions, including 
instructional assistance, will be accomplished in Phase III. This 
deferement is being made for two reasons. One, this is in itself a major 
task, and were it done in Phase IIB, the Implementation of a prototype 
system would be delayed, and it is CCN*s desire to begin having operational, 
evaluative experience as soon as is feasible. Two, it is expected that 
many users will be interested initially in sequential searching (most 
current data bases are on magnetic tape) and the interest in interactive 
processing will grow slowly as the batch system is being used. The data 
bases to be processed by the prototype will be specified during Phase IIB 
and as they become available will be implemented.. This should be a minor 
task since only a read routine need be written when a new base is to be 
used. It is intended, however, to have the prototype system take over 









the work of the experimental systems (described below) , which includes 
searching of the Chemical Abstracts Services tapes. 



General Design Approach 

Design guidelines established in Phase IIA include the followings 

a. The design of the IPS must ensure modularity at all levels 
of operation, including services, user request handling, 
intra-service operation, and support functions (such as 
monitoring and accounting) , 

b. All input-output functions must be isolated. This will 
ensure not only data-base independence, but also machine- 
configuration independence, insofar as possible, 

c. All operating system functions must be isolated; that is 
the IPS should not be dependent upon particular versions 
of the operating system, 

d. Plans must be made for a time-sharing facility and remote 
job entry. It is expected that use will be made of IBM's 
Time Sharing Option when it is released. Provisions will 
be made for the remote entry of jobs from a peripheral 
computer located in the Library, 

e. An independent, functional component of the IPS will provide 
interactive assistance for the on-line user, ILR already has 
developed such a program, called DISCUS, and we shall con- 
sider the use of that existing program. 



EXPERIMENTAL OPERATIONS 

In parallel with the designing, documentation, and prototype 
development CCN will conduct experimental operations with interim 
software to obtain valuable information concerning real-time operational 
problems and procedures, evaluation and monitoring techniques, user 
interface problems, and the quantity and quality of services needed. 

This information] will have considerable influence on the detailed design 
and implementation of all parts of the prototype system. Both ILR and 
the Library will help in providing experimental services to the user 
community. 

Most of the experimental operations will center around the use of 
IBM r sTEXT-PAC system in providing SD1 and retrospective searches of the 
Chemical Abstracts Service CA-Condensates File. As other data bases be- 
come available, TEXT-PAC will be used to process them. It is planned that 
the prototype system will take over this work before the end of Phase IIB 
(see . Figure ' 7 ) i ; . 



During Phase IIB, the project will get practical experience with 
numerical processing by experimenting with the tapes from the 1970 
Census, CCN has met with SCRIS (Southern California Regional Information 
Study) to consider using their programs as interim software; other 
solutions also are being explored (such as contacts with DUft'Labs, Inc, 
through the Library’s participation in a CRL-Ford Foundation cooperative 
project) „ The UCLA Library has had extensive discussions on all aspects 
of the Census and will be working closely with CCN. The modular retrieval 
program developed in Phase XIA (see Part 1 of Phase IIA Final Report) also 
will be used to provide searching, indexing, and processing of the ERIC 
file for a selected group of users to acquire additional data to guide 
prototype development. 



COORDINATION AND COMMUNICATION 
Coordination 

ILR will follow closely the progress being made on the tasks being 
conducted by CCN and the Library, This will be done by personal contact 
with the respective staffs, by discussions in both formal and informal 
meetings, and by monitoring the documentation produced by the two groups. 
Coordination will be enhanced by suggesting ideas to CCN and the Library 
based on the experience and expertise of the ILR staff, and by filtering 
inputs from user committees and other CIS committees (see below) to 
appropriate members of CCN and the Library, ILR will assist the Principal 
Investigator by preparing the quarterly progress reports submitted to NSF 
and will monitor and summarize cost data for inclusion in the reports. 

ILR also will assist with system test and evaluation activities through 
review and consultation (but is not staffed to perforin a 2JL tasks outlined 
in the Phase XIB Supplement to the Phase II Proposal for the Development 
of a Center for Information Services) , 

Communication 

Several techniques will be used in Phase IIB and succeeding phases 
to strengthen communications with the user community, to inform and 
involve library personnel in CIS activities including design support and 
procedure preparation, to interface with other University of California 
libraries, data sources, and developments, and to interact with other 
data base and computer program resource locations. Seminars based upon 
those held in Phase IIA will be conducted, action will be taken to locate 
data bases and determine user interest, and user committees will be asked 
to increase their level of involvement. 



SEMINAR FOR LIBRARY STAFF 

The evidence from Phase IIA indicates decisively that the idea of 
mounting seminars for working librarians is a good one and worth further 



Investigation as a permanent part of the library’s involvement in the 
field of mechanized information services. 

The seminars held in Phase IIA had a twofold purpose: 

a. To acquaint library staff with the nature and scope of 
machine readable information, and with the implications 
of having the library act as the responsible agency for 
acquiring and providing services from them. 

b. To have the librarians act as a body of consultants, 
providing the project staff with qualified input towards 
the task of designing specifications for a system that 
will be viable for the UCLA Library environment. 

Phase IIA seminar experiences are discussed in considerable detail in 
Part 6 of this Final Report 

In Phase IIB, it is foreseen that there will be a need for at least 
one and possibly two more such seminars, though their function may by 
then have changed somewhat (the work of specification being essentially 
complete) , Bearing in mind that there is a growing demand for this type 
of seminar to be given in other libraries, there are a number of suggested 
directions that such seminars could take: 

a. The 10-week presentation could be condensed to 2 weeks 
(full time attendance) « 

b. The 2 -week format might in turn be condensed to a crash 
program of, say, 2 days, 

c. In this version, the seminar might then become a travelling 
presentation, rather than fixed at UCLA, 

d. Or, the lectures might be put on videotape and distributed. 

Any one of the above configurations might best be handled by some 
agency other than the UCLA Library (e.g, the University Extension or a 
commercial organization) in which case the Library would just be one of 
a number of organizations that sent participants- It may be, however, 
that the Library has sufficient need for the seminars on a permanent 
basis to continue to run its own, A shortened version of the seminar 
could continue in use as a medium of instruction for Library Staff; to 
provide an introduction to the Library’s range of data base holdings 
and services for new graduate students, interested faculty, committees, 
etc, * and to provide a forum for Library Staff to meet with representa- 
tives of data bases potentially suitable for acquisition, by incorporating 
a guest presentation into the standing contents of the seminar. 



USER NEEDS 



V. USER AND OTHER PROJECT COMMITTEES 



CIS development is focused on the needs of the user. On one hand, 
the user’s needs must be met by involving him in the planning and develop- 
ment of the system. On the other hand, the system must be created before 
the user can confidently project actual needs. This latter restriction 
must be faced in the development of all information processing systems. 

The user must actually use the new tools and techniques in order to 
reformat his intellectual attitudes and traditional approaches. Then 
with experience and understanding he can better define current and future 
requirements — a situation often calling for system redesign and update. 

I n this sense the CIS development must be viewed as an ongoing activity 
that continually strives to anticipate and met changing demands, require- 
ments, and environments. 



A system that acquires data but is not responsive to user needs 
serves only an archival function. By contrast, CIS is user oriented and 
provides, or interfaces with, those functional activities required to help 
the user locate appropriate data bases and search for and acquire needed 
information. To meet these needs, the system must develop procedures for 
responding to user demands. The process of acquiring and housing data bases 
will sometimes require that CIS serve as a resource collection point before 
user pressure is felt. Users must be made aware of available resources, 
tools, and procedures through planned activities and announcements. 



COMMITTEES 



A number of committees were established on the UCLA campus and within 
the UC system during Phase IIA to guide CIS development. Three policy 
committees (the Management Committee, the Library Liaison Committee, and 
the Extra-Campus Committee) provide advice on policy issues, the several 
technical committees (not all appointed) continue to address technical 
issues such as text processing, file management, hardware evaluation, and 
O^Ilp^ppcefW.ng 5 ; ;v.and-. title User Committees not only provide information, 
guidance, and support but also participate actively in data base selection 
and, where possibles in the experimental utilization and evaluation of 
system services. Other committees established to consider data base 
problems are also deemed to be valuable CIS resources . 



Membership 



Committee membership as recorded during Phase I IA is shown by Figure 8. 
During Phase IIB, however, the composition of such committees will vary to 



FIGURE 8 



C.I.8. Committees 
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April 1970 



POLICY COMMITTEES 



1. Management Committee 



R. M. Hayes, Director, ILR 
W. B. Kehl, Director, CCM 
R* G. Vesper, University Librarian 

2 . Library Liaison Committee 

Paul Miles, Assistant University Librarian (Chairman) 

Everett T, Moore, Assistant University Librarian 
Anthony Hall, Head, Library Systems Staff 
Mary Ryan, Head, Public Affairs Service 
Robert L. Collison, Head, Reference Department 
Lorraine Mathias, Head, Education and Psychology Library 
Johanna Tallman, Head, Engineering and Math— Sciences Library 
Frederick E* Smith, Head, Law Library 
Nelson Gilman, Assistant Biomedical Librarian 
Judith Cor in. Head, Physics Library 
Pat Walter, Brain Information Service 

3, Extra-Campus Committee 

Anthony Hall, Head, Library Systems Staff, UCLA (Chairman) 
Herb Ahn, Head, Library Systems Analysis Office, Irvine 
Scott Buginas, Library Manager, Lawrence Radiation 
Laboratory, Livermore 

Kay Forrest, Assistant University Librarian for the 
Sciences, Riverside ■ 

Ming-Yu Li, Documentation Specialist, Davis 

TECHNICAL COMMITTEES 

1, Text Processing Committee 

Professor Harold Borko, School of Library Service (Chairman) 
Professor Vinton Dearing, Department of English 
Professor David Packard ^Department of Classics 
Martin; Kaye , Department of" Linguistics ; . '> ' 
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2 » File Management Committee 
To be announced 

3 , Hardware Evaluation Committee 



To be announced 

4, On-Line Processing Committee 
To be announced 
USER COMMITTEES 

1. Social Sciences User Committee 

Professor Dwaine Marvick, Department of Political Science, 
Chairman 

Professor Marvin Hoffenberg, Department of Political Science 
Professor Carl Hensler, Department of Political Science 
Professor Peter Kamnitzer, School of Architecture & Urban 
Planning 

Mary Ryan, Head, Public Affairs Service 

2 . Education and Psychology User Committee 



Dr, Lorraine Mathies, Head, Education and Psychology 
Library, Chairman 

Professor Arthur Cohen, ERIC Clearinghouse for Junior 
Colleges 

Professor Jerrold Novotney, Graduate School of Education, 
: (representing Dean John Goodlad) 

Professor Marvin Aik in. Center for the Evaluation of 
Instruction 

Professor Kent Dallett, Department of Psychology 
Professor Ronald Freeman, Department of English 

3 . Physical Sciences and Engineering User Committee 

Johanna Tallman, Head, EMS Library 
Professor George Igo, Physics 
Professor Ronald Shreve, Geology 
Professor Rointan Bunshah, Engineering 
Professor Frank Spaid, Engineering 
Professor David Evans ,, Chemistry 
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FIGURE 8 (continued) 



4, Life Sciences User Committee 



Louise Darling, Head., Biomedical Library, Chairman 
Professor- Peter Amacher, Medical History and Physiology 
Professor John Belkin, Zoology 

Professor Donald Jenden, Pharmacology and Biomathematios 
Professor Frank Massey, Biostatistics and Biomathematics 
Professor Richard Riley, Radiology 

5« Law User Committee 

Frederick E, Smith, Head, Law Library, Chairman 
Professor Reginald Alleyne, Law 
Professor Lawrence Sager, Law 

D. OTHER 

1. University Librarians 

Faculty Committee on the 1970 Census 
Dean Harold Somers, Economics 

Dean Harvey Perloff , Architecture and Urban Planning 
Dean L, S, Georke, Public Health 

Professor Robert M, Hayes, Institute of Library Research 
Professor George Sabagh, Sociology 

Professor Werner Hirsch, Institute of Government and Public 
Affairs 

Professor Bertram Bussell, Engineering 
Professor Marvin Hoffenberg, Political Science 
Professor Dwaine Marvick, Political Science 
Professor Y* P. Chen, Economics 
Mr. William Kehl, Campus Computing Network 

Working Subcommittee on Policies for Procurement and Use of 
the Census 

Professor Bertram Bussell, Chairman 
Professor Y. P. Chen 
Professor Robert M. Hayes / v 

- Mr. William Kehl •’■■I •• 

Professor Dwaine Marviek 
Professor Frank Massey 
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reflect personnel changes and new assignments , For example, the Management 
Committee composition for Phase IIB (1/71 - 6/72) is: 

CIS Phase IIB Management Committee 
William B. Kehl, Director CCN, (Chairman) 

Robert G, Vosper, University Librarian 
Harold Berko, Professor,* (Advisor to ILR) 

Robert L. Carmichael, Manager ILR 

Robert M, Hayes, Professor,* (Advisor to CIS) 

Several members of the Library Liaison Committee, themselves heads of 
UCLA campus libraries or assistant university librarians, serve as chairmen 
of the identified User Committees. Other committees are headed by inter- 
ested and qualified members of the academic staff. Committee members are 
drawn from these two groups. The User Committee also serves to link CIS 
with the traditional departmental faculty -membe red library committees. 

Utilization 

User committees are indoctrinated by CIS and library personnel: 
first, so that committee members can provide sound guidance to the UCLA 
Library in the form of information leading to an explicit program for the 
acquisition of data bases; secondly, so that the committees can provide 
input needed to generate a list of priority-ordered experimental services 
to guide CIS development planning and experimentation by project personnel; 
third, to provide communication links to their departments to help identify 
the users; and fourth, so that eventually they will become an advisory body 
for the operational system. 

Throughout the project, user committees will be utilized to support* 

‘System Development 
"Data Base Acquisition 
'Data Base Utilization 
>;// and Ihdpctrination 



‘System Test and Evaluation 

They will be called upon to support system development by helping 
identify requirements, procedures , and guidelines ; and by participation 
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m and evaluation of experimental services. They will help acquire data 
bases through the processes of identifying specific data bases needed for 
teaching, research, or grant support* participate as needed in the actual 
negotiation process for acquisition; and help locate data bases owned by 
individuals in the departments and arrange for these data bases or master 
copies to be deposited in the CIS inventory--thus making such resources 
available to a wider audience of potential users. In addition to data 
bases formally acquired, considerable use will be made of user committees 
in helping determine which new data bases-“Oreated by processing already 
acquired data bases-— are to be designated and handled as system resources. 
User committees will support data base utilization in at least three ways: 
exposure to search profile generation procedures so as to become better 
acquainted with system techniques* planned use of data bases as teaching 
aids or for class assigned activities; and for members of the committee to 
initiate programs of planned research— —either as an individual investiga- 
tion, intramurally funded research, or extramurally funded grants or 
contracts. 

The user committees will be used to implement the announcement and 
indoctrinations functions. These traditional library services, thus 
augmented, are needed to insure that potential users are made aware of 
the Center for Information Services, its capabilities, its resources and 
its procedures. Another most important way in which user committees will 
support CIS development is to participate in system test and system 
evaluation activities. These two items insure first, that CIS as imple- 
mented is capable of servicing user needs in a satisfactory manner; 
secondly, through evaluation, areas of the system needing improvement 
are identified. User committees also will be used where appropriate to 
strengthen ties with users and user committees established on other 
University of California campuses. 



VI, DATA BASE CONSIDERATIONS 



BACKGROUND 

A major part of the work of specifying procedures by which the 
library can handle machine-readable data bases consists of supplying an 
answer to the question ’’How much concession shall be made to the form, 
the medium, in which these information stores are appearing?" By and 
large it is the form in which computerized data Is embodied that gives 
the library a problem. The actual information, as was recognized several 
years ago, Is relatively easier to deal with, for in most cases it is 
information that has an existing counterpart in print (e.f. Chemical 
Abstracts, MEDLARS, ERIC and many others). Furthermore, the work of 
Lubetzky, as reflected in the 1967 Anglo-American code of cataloging 
rules, has made this distinction an overt one In general library prac- 
tice, to the benefit of all concerned. 

To gain a perspective on the way the UCLA Library system responds 
to differences in the medium of information transmission, it is necessary 
to remember that the Library is presently receiving three main types or 
classes of material — books, serials, and microforms. And the technical 
processing procedures are structured around their form rather than their 
content. In the UCLA University Research Library, for example, the 
Acquisitions section (which deals primarily with books and monographs) 
has a specific sub-unit for microforms, while serials publications are 
processed by a Serials Department. Cataloging is integrated into a 
single department, but it has clearly identified sub-units for different 
forms of material. Likewise, public service aspects are to a degree 
treated according to the medium of publication, with separate service 
points for serials and microforms. 



APPROACHES 

One possible solution to the problem of how the Library shall process 
machine readable data bases is immediately evident-- it could be done on 
the basis of form. This would in fact be consonant with the trends just 
outlined. Particularly if one looks at the colossal development of 
periodical publications in the last 50 years 9 tnere are reasonable 
grounds for assuming that a .separate unit within the Library, oriented 
to this particular medium, may become the ultimate, settled solution. 
However, there are many pressing reasons not to begin the integration of 
tapes into the Library by taking that assumption as fact. In the first 
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place, to establish entire new units (UCLA is effectively a system of 19 
libraries with three primary nodes) would be enormously expensive, without 
any guarantee of added efficiency or competence. Second, it would be an 
obvious tactical error to impose on the library a new set of procedures, 
specified from the outside by a professional research group. Third, and 
undoubtedly more important in the long term, to do so would be implicitly 
to admit that machine-readable data bases cannot in fact be dealt with by 
existing library procedures, or any viable modification of them, and that 
existing library skills are, for all practical purposes, useless. This 
comes close to negating the basic postulate of this project that the 
library jls the agency best fitted to take this responsibility precisely 
because of the information handling expertise of its staff. There are 
other contributory factors which need not be explored here, such as the 
probable effect this would have of confirming some librarians f worst fears 
about ’the computer*. 

Conversely, it is not to be expected that the best method of solving 
the many complex questions of data base servicing will be to make jio eon- 
cessions to the new and (to librarians) unfamiliar format. To attempt to 
rely, for example, wholly on the existing 10-part order form (3x5 inches) 
for acquisition would be to invite a number of sizeable problems, both 
administrative (clerical) and intellectual (bibliographic) and implicitly 
to deny that data bases are a special phenomenon demanding a carefully 
phased program of research and development; obviously this too would be 
counter to the entire thrust of the CIS project since its inception in 1966 

It appears that the most rational operational patterns for handling 
tapes in the near future will be centered upon the practising librarians, 
with support (on a quasi-consultative basis) from members of the CIS 
project. And this function is expected to decline once this particular 
historical situation is past, and library schools have sent out several 
generations of students with a thorough grounding in mechanized informa- 
tion processing. But for the present, the various novel features of 
computerized information storage and retrieval will mean establishing 
shared work patterns, probably on a basis of day-to-day co-operation, 
between CIS personnel in the Library (operating initially through the 
Systems Department) and librarians in the relevant areas of acquisition, 
cataloging, reference and perhaps also circulation. The two seminars 
held during 1970 ensure that at least one individual (usually more) in 
each of the various departments has had some exposure to data bases and 
the handling of mechanized information. At this time it is sufficient to 
note that the basic pattern of close co-operation between project staff 
and UCLA librarians, with the underlying emphasis on the latter, will be 
taken as understood during the discussion of the following tasks. 



TRIAL ACQUISITION OF DATA BASES 

During 1971 the Center for Information Services will work closely 
with the UCLA Library staff to establish procedures, criteria, and 
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guidelines for acquiring the first data bases for actual service. Heavy 
• reliance will be placed on available information, faculty judgment, re- 

lated University developments, and the nature of the file. 

Existing Bibliographic Information 

Since no bibliographic network comparable to that used in book selec- 
tion yet exists for machine readable data bases, this information will in 
general be drawn from files in the Institute of Library Research, These 
have been maintained for over three years with precisely this end in 
view--to provide a body of such information as does exist (publishers’ 
announcements, clippings from the professional journals, sample search 
forms, lease or purchase agreements, personal letters, and any other 
bibliographic data that is available, plus some technical documentation) 
on which to make intelligent acquisition decisions, 

I 

| 

V 

| Judgment of Faculty 

As described in the Third Quarterly Progress Report on Phase IIA 
(April 1970) of this project, and in the User Committee section of this 
document, a series of CIS User Committees has been established as a 
vehicle for broad-based communication with the entire UCLA faculty and 
their needs, preferences, and expectations regarding both the informational 
content of data bases and the range of services to be provided from them. 

To ensure close liaison at a senior level, the Head of the disciplinary 
library serves as a member (usually. In fact, as the Chairman) of the 
respective CIS User Committee, 

University wide Situations 

Related developments already underway at several University of 
California campuses make it imperative that the acquisition of data 
bases be viewed in the context of the whole University, For example, 
the Library at UC Riverside has been providing an experimental SDI 
service from Chemical Abstracts tapes for over a year, and is planning 
to extend Its coverage to Biological Abstracts in the near future. The 
Library at UC Davis, a campus with strong research priorities towards 
the agricultural sciences, is actively exploring the acquisition of the 
National Agricultural Library’s CAIN tapes that contain catalog data. At 
UC Berkeley, the Institute of Governmental Studies and the Survey Research 
Center have jointly agreed to form a Summary Tape Processing Center for 
the 1970 Census files. Of these and similar developments in the UC system, 
the selection decision for UCLA’s Center for Information Services will 
have to take account. This should not limit our freedom of choice, nor 
need it place constraints on the design of a CIS for UCLA — but it will 
serve to ensure that large sums of money are not expended in unnecessary 
duplication of resources, especially where UCLA’s usage of a particular 
file can be projected to be minimal in the immediate future (e.g, the 
CAIN tapes) . 
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Mature of File 



The Center for Information Services is being created with the inten- 
tion that it will eventually become a library service for the whole campus 
community at UCLA. As such it is being designed to handle all of the three 
major types of data base currently being produced--bibliographic reference 
files , numerical data files, and full text files. At this stage, however, 
it is proposed to continue to focus our efforts upon the two primary 
subsets; scientific and technical data bases, and bibliographic reference 
data. While not excluding other data bases for which a pressing need may 
occur, we anticipate that by the end of 1971, the CIS should be in a 
position to offer experimental services from a modest set of files that 
will include some of the following data bases; 

Chemical Abstracts 
Biological Abstracts 

American Institute of Physics' SPIN tapes 
Institute of Electrical and Electronic Engineers’ 

REFLECS tapes 
MEDLARS 

Engineering Index 

Clearinghouse for Federal Scientific and Technical 
Information reference tapes (USGRDR) 

Psychological Abstracts 
Nuclear Science Abstracts 
U.S. Census of 1970 
Dissertation Abstracts 
ERIC (RIB and CUE) 

However, in order to maximize the chances of mounting a viable test service 
at a reasonably early date (i.e. in calendar 1971) it is foreseen that 
another criterion might be influential upon selection, namely the technical 
difficulty of the file. For example, if the faculty have to choose between 
two fairly costly data bases, each of more or less equal usefulness, the 
degree of programming difficulty (of a complex file like Nuclear Science 
Abstracts, for instance) might then become a factor for consideration. 



TRIAL CATALOGING OF DATA BASES 

Cataloging will be accomplished according to routines developed during 
Phase XIA. Preliminary findings from the CIS Seminars for library staff 
indicate that librarians believe that cataloging procedures for data bases 
can be closely integrated with the regular cataloging workflow, and that 
they desire that this be the situation. Amongst other things, this will 
entail the provision of a tape "read" for use by the cataloging staff* 
the utilization where possible of the Anglo— American cataloging rules; 
and the appearance of catalog data in the central (Research Library) ’ 
catalog plus whichever branch library may be most directly affected by 
service requests for a particular data base. Formal procedures will be 
documented in Phase IIB* ! v ' .’••• 
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TRIAL SERVICE FROM DATA BASES 



Service to users is a serious concern of CIS and much remains to be 
done, not only in providing services but in defining the services to be 
provided. The following paragraphs illustrate some of the issues and 
approaches considered to date. 

During Phase IIA, a prototype tape lending service was initiated on 
an jid hoc basis by the Library to cope with user needs which were already 
arising in advance of the Library's ability to offer a processing or 
CIS-type service. The item in question was the 1970 Census Test tape 
(first count) with which ILR and the Library Systems staff had been ex- 
perimenting, The procedure was quite simple; the data was copied for the 
borrower and the returned tape treated as a scratch tape. Solutions such 
as this will be investigated in Phase IIB, not because it is a proposed 
design solution for CIS, but because It does fill a definite temporary need 
and was created out of necessity. What is not known is how temporary, or 
conversely, how permanent, this type of service is destined to be. Is the 
demand to "borrow tapes" in fact likely to atrophy with the advent of CIS 
processing services, or to persist? If it persists, what are the causes? 

Do the clients need types or levels of processing on the tapes which CIS 
cannot provide? If so, what are they, and why not, etc,? Will such a 
lending service lead to an unbearable tape copying burden on the Library's 
computer? Will there be an unacceptable copyright problem involved in 
supplying copies? What are the copying costs, and what would be a 
realistic copying charge? These and other questions will only he answered 
by actual experience. 

From the project viewpoint, it Is Intended that the processing ser- 
vices offered by CIS be announced and made available by the Library as 
they are developed and debugged by CIS technical staff. In Phase IIB, 
the reference librarians will work closely with CIS library staff to 
analyze incoming requests, transcribe them into a canonical form, and then 
scrutinize the output and maintain contact with the user to obtain regular 
and detailed feedback. The preliminary set of forms for the user-librarian 
interface is currently being designed; obviously, a most important part of 
the librarians' work in this phase will be to test the suitability of these 
forms, and recommend changes as necessary. Members of the CIS User 
Committees, being the best generally informed segments of the faculty, will 
be among the first serious users of the system, and their feedback will 
be a critical indicator of the success of the initial services. System 
monitoring procedures will be developed and clarified, and close liaison 
will be maintained between project staff assigned to the Library and those 
assigned to the Campus Computing Network. 
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which are necessary for the UCLA Library T s bookkeeping operation will be 
completed and filed in the present manner and at the appropriate times. 

For example, the payment from the CIS budget to the Library’s budget for 
services or acquisition cost could be handled as two transactions — January 
1971 for the latter half of FY 70-71 and July 1971 for FY 71-72, 

No reliable guide exists as to when, and how, the technical documen- 
tation of the file will arrive: it has been known to precede, accompany 
or follow the file, in sections or as a whole. Often it is updated by 
loose-leaf sheets that might be received at any time. These documents 
will need to be accession stamped, and recorded in the Tape Documentation 
file. The CIS programming staff, either at CCN or in the Library, will 
need to use them forthwith. After a suitable interval it may be made 
available to the public on demand, for on-the-spot consultation only or 
for copying purposes. Other ideas to be further explored are: procuring, 

from the manufacturer or by Xeroxing, a standard number of copies of the 
documentation for deposit in specific locations (e.g., URL, CCN, a branch 
library) ; and cataloging the documentation as regular library material. 

Subscription on a Serial Basis 



This procedure n?ill be used to acquire the abstracting and indexing 
serials in magnetic tape form; also, occasionally, actual journals, such 
as the Journal of Chemical Documentation , The procedure in many cases 
will follow the basic sequence for straightforward purchase, with two 
main variations that will need to be investigated in Phase I IB . 

The first variation concerns the integration of the acquisitions 
(and, to a lesser, extent, cataloging) records with those of the existing 
Serials Department, The basic question, which is not yet solved, but 
of which the UCLA Library is now aware, is "would serials tapes be better 
handled by the Serials Department, with its claims procedures, its dis- 
tinct set of forms, and its existing control over the printed counterparts 
of data bases such as Chemical Abstracts , Engineering Index , etc,?" 

Whether such advantages would be offset by the necessity of establishing 
two sets of acquisitions procedures for tapes needs investigation. 

The second area of concern is that of lease agreements, such as are 
now operative for Chemical Abstracts , IEEE REFLECS tapes, etc. Joint 
procedures will have to be evolved to enable the Library Administrative 
Office to scrutinize suggested lease agreements before any firm commitment 
to acquisition is made. The eoncept of single-point acquisition by any 
one of the nine UC campuses will be further explored , At the moment it 
appears that some of the agreements being demanded by the manufacturer 
embody several new precedents regarding copyright and ownership. Close 
examination of all lease agreements is therefore especially necessary 
before the Library can obligate itself to accept, and maintain ongoing 
services, ■ - • % 



Service from a Remote Stat ion 



This type of access takes account of the fact that some data bases 
from which UCLA scholars will require a service may not be available for 
actual possession by the CIS through purchase or subscription, an example 
being the Dissertation Abstracts DATRIX service,, A service arrangement 
—2 thus necessary, which at present customarily involves batch access to 
the file, with the owners accepting formulated or semi-formulated search 
requests, retaining exclusive control of both the data and the programs 
by which they will search it, and mailing to the client the printed 
results of the search. In this sense, the possibilities of go— operative 
service between UCLA and other UC campuses will continue to be explored-- 
this will have value both in and for itself,, and also as a model for 
future developments in this direction, either within the UC system or 
outside it. Because in principle it seems obvious that the user of CIS 
services should not have to concern himself with where the data might 
happen to be located, procedures will need to be developed to enable 
reference librarians to act as the interface just as they now do for 
interlibrary loans. Within a relatively short period, reference librarians 
should be able to handle on-line requests through the use of a terminal 
in the library (the UCLA Biomedical Library already has a CRT terminal for 
on-line querying of its own serial holdings) , 

On-Line Access to a Remote Facility 

For some operations, this advance in access technology will replace 
the approach described above where the distant owner of the remote data 
base processes the request. In other instances, the remote data base 
will remain inviolate as far as remote query is concerned. With regard 
to on-line access and remote query, only a few working examples of this 
mode of access can presently be cited; the UCLA MEDLARS Station now has 
direct access by teletypewriter to the Abridged Index Medicus file stored 
at the System Development Corporation in Santa Monica (the AIM-TWX System) , 
This is not strictly a matter of direct concern to CIS because the MEDLARS 
operation is a separate entity funded from other sources. The CIS respon- 
sibility exists to the extent of establishing an interface with the MEDLARS 
stat ion- -by what techniques the patron is then served may remain outside 
the CIS frame of reference. Initially this is how cooperation within the 
UC system or the local geographic area will evolve, although a long range 
goal is certainly the ability to tap into or participate in some kind of 
national network such as is envisioned by EDUCQM, During Phase IIB, the 
prospects for converting some of the remote batch operations into on-line 
file manipulation must be investigated, 

INVENTORY PROCEDURES FOR ACQUIRED DATA BASES 
Record s 

Current plans call for two parallel sets of records to be kept — one 
bv the Acquisitions Department, to match as closely as possible its 



existing files, and one by the CIS office. Investigation will be made of 
the alternative strengths and weaknesses of various systems of forms 
generation and control, including the possibility of creating one machine- 
road able master record from which any others could be drawn. The 
postulated set of files to be maintained by CIS is described below, and 
the data elements that might appear on forms in those files are listed in 
Figure 9. 

Proposed CIS Files 

The files tentatively identified during Phase XIA are described 
below. 

Data Base File . Consisting of all the publicity and informational 
literature about all of the data bases that can be traced. Later, a 
separate file might be established for overseas data bases, organized by 
the given name of the data base. 

"On Order” File . Capable of accommodating about 100 entries; the 
opening operational phases will see this fill rapidly, but for some years 
to come it is doubtful whether more than 100 data bases will be on order 
at any one time. Organized by the given name of the data base, 

"Item Received” File , Contains one copy of the order form and invoic- 

ing for every completed transaction; initially with space for 500 entries. 
Organized by the given name of the data base. 

Vendor File . Contains space for 500 entries. This will take longer 
to fill than an "item received" file of the same capacity — but there is 
no logical reason why each data base should not have a different manu- 
facturer. Where the manufacturer is not also the vendor (i . e . , where a 

dealer is involved) the entry should be under the vendor with 

cross-reference from the manufacturer. It should contain the postal 
address of the vendor, including the names and addresses of hia key 
personnel associated with sales of data bases (e.g 8 , his West Coast 
representative). Under this, a brief statement of any discount or 
approval or other service arrangements, followed by a chronological 
listing of the data bases we have purchased from him, showing cost. 

Tape Documentation Fil e. This file contains: 

a. a copy of the original documentation accompanying the tape, 

b. a report by the CIS staff member responsible for initial 
work su-.u as opening the tape and identifying the elements 
of its structure, 

c. any relevant printout associated with /(b) , e.g. , the dump, 
the reiacL routine (plus the card deck) and a specimen of the 
correctly formatted printout. 
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PROPOSED CIS FILES 
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CIS Files 


1 Master 

File 


w 
fed/ 
d / 
•H; 
nu/ 0) 

fH r-A 

a *h 

CO 

CJ 


r 

On Order 1 

File j 


i 

Item Received 1 
File 


a oj 

TD I— 1 

C *H 
cu Cm 
> 


Public 

Catalog 


Manufacturer 


X 


X 


X 


X 


X 


X 


Address 


X 


X 


X 


X 


X 


X 


Issuing Agency (if different) 
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Title of data base 
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Year of issue 
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Lease or Purchase 


X 


X 


X 


X 






Basic price 


X 


X 


X 


X 






Other Factors (Discount?) 


X 


X 


X 


X 






Fund 


X 


X 


X 


X 






Authorized by 


X 


X 


X 


X 






Countersigned by 


X 


X 


X 


X 






Order timber 


X 


X 


X 


X 






Makers documentation 


X 


X 


X 


X 






Equipment on which created 


X 












Dens ity (b . p , i . ) 


X 












Tracks, parity 


X 












Recording mode (BCD, EBCDIC) 


X 












Record format 


X 












Block length 


X 












Tape label (s) 


X 












Call number 


X 


X 








X 


Subject headings 


X 


X 








X 


UCLA holdings 


X 


X 








X 


Programs available 


X 


X 








X 


Copying restrictions 


X 


X 








X 


Remarks 




X 


X 


X 


X 


X 



O 




d. any technical correspondence resulting from (b) , e.g., to 
the owner or to another UC campus, itemizing gaps in the 
original documentation, etc., 

e. later work as it occurs, roughly in the same mode, i-. e . , 
a report of what was done or attempted plus some sample 
printout. Any relevant contributions by users should be 
included, with acknowledgments and references. 

In addition to the above files, the CIS as a whole will need a publicly 
available record analogous to the serials list. This file, again with space 
for about 500 entries initially, would be created after the cataloging is 
all completed and the data base announced for public use; the aim would be 
to have an alphabetic list of the data bases in the system that would show 
all necessary details of both physical and intellectual location. It might 
be called the CIS holdings file, and it would be the working record which the 
reference librarian would have immediately available to help meet inquiries. 
In other words , It would provide a means of avoiding the chore of looking 
through the whole of a card catalog, though derived directly therefrom. 



AVAILABLE DATA BASES INVENTORY PROCEDURES 

The task of compiling an inventory of available data bases, with empha- , 
sis on those most likely to be acquired by a library-based Center for 
Information Services, was one of the most useful components of the Phase I 
study of this project, as the continuing demand for that segment of the Final 
Report amply confirms.* Some updating has since been carried out for 
internal project information in Phase IIA. Two desiderata now appear: 

a. The Inventory of Available Data Bases should be thoroughly 
updated on a national scale by actively approaching manu- 
facturers for current information, and should then be inserted 
(as a file) into the Library as the basic bibliographic tool 
for CIS acquisitions. 

b. An inventory should be compiled of all data bases currently 
held (by any agency) at UCLA, and then a similar project should 
cover the whole UC system, thus generating in effect a union 
catalogue. It has been suggested that the CIS Extra-Campus 
Committee would be an appropriate body to undertake this 
latter task, A draft form prepared by this committee in 
anticipation of such an effort Is shown on Figure 10. 

The procedures by which the inventory data will be collected and maintained 
will be finalized during Phase I IB. 



’’Inventory of Available Data Bases” by Joan C. Troutman. Part 4 
of the Final Report on Mechanized Information Services in the 
University Library. Los Angeles, ILR, December 1967. 
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FIGURE 10 

DATA BASE QUESTIONNAIRE 



Sheet 1 



NAME OF DATA BASE: 

SUPPLIER; 

ADDRESS * 

CONTENT: 



/ 




/ 

/ 



SUBJECT COVERAGE; 
TYPE OF DATA; 



FORM : 
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□ 
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Programming 1 anguage : 
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NO 



NO 
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DATA BASE QUEST I 0 NNA IRE 
Con t 1 d 



UPlGJN: Owned fay* 



FIGURE IQ Contimipri 
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Created on 
our campus 



a 

□ 



Annua 1 ] ease $ 



Purchase price $ 
Restrictions on use;' 



HOLDINGS: Wher 



e stored; 



Currently subscribing? YES 
Number of years of backfire 
Number of reels in backfile 



NO 



USER INFORMATION: 

Availability (To whom available) 
Can be copied? ygg 

Copy cost: $ 

D 



NO 



r e c t inqui r i 



es to: 



Name : 



Departmen t : 
Campus : 



Authorship (If locally d 
Number of users 
Are all u 



Telephone 



y developed): 



isers on your campus? YES 



If 



NO 



n °t, please list users' organization and locations 



Please return 



this form to: Name; 



Depa r tmen t : 
Campus • 



Telephone: 
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VIII. 



DOCUMENTATION CATEGORIES 



IDENTIFICATION 



The development phase of the CIS system! from concept through imple- 
mentation requires that certain individual products (computer programs, 
hardware, and documents) be identified and prepared or procured in 
accordance with agreed upon schedules. These products, often referred to 
as "end items", represent specific expenditures of time, manpower, and 
money. Such parameters can be expressed on a PERT or Milestone chart 
and the associated tasks and rates of accomplishment can be tracked 
manually or automatically by computer. The problems of accounting are 
significantly less complex than are the judgments that determine the 
assigning of values such as degree of completion or accomplishment , 
level of difficulty, or impact of failure. 

A good system of documentation is absolutely essential. Four 
important categories of documents are- 

a. System Specification, System Testing, and System Description 
documents . 

b. End Item Specif ieation and Testing documents, 

c. Supporting Documentation which encompasses procedures, 
standards, guides, handbooks, and manuals. 

d. Development Schedules, Progress Reporting, and Planning 
documents. 

Documents such as the above are not developed by surmise, conjecture, 
or conceptual thinking alone. They serve to guide l well as record the 
results of a significant array of activities including inquiry, design, 
construction, test, operation, comparison, examination, and experimenta- 
tion, These and related activities are initiated or conducted as a result 
of, or in support of, system analysis, operational evaluation, research 
studies, system synthesis, and concept development. From these efforts 
come the information and understanding that is used to create specific 
system requirements, design specifications, tested design solutions, and 
operational and maintenance guidelines. Properly organized and recorded, 
the resulting set of documents provides the needed insight and guidance 
to identify and pursue serious system development, maintenance, and 
improvement tasks. 
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Specification documents are written to record: performance and 

design requirements for the identified functions and functional relation- 
ships; the detailed design solutions implemented; and the restrictions 
and limitations within which the system is intended to operate. 

Testing documents record the test plan, test procedure, and generated 
test results. Tests are conducted at the item level, the subsystem or 
modular assembly level, and the system level. Tests are intended to 
verify that the item or assembly or system performs to the level pre- 
scribed by the related specification. Such tests are prescriptive by 
nature and are performed in a controlled environment. Their goal is to 
determine that acceptance levels are met, and that the required 
documentation for the specified hardware and computer program products 
agrees with the items as produced. 

Systems evaluation is performed on an operational system. in an 
uncontrolled (but measured) environment to determine the actual capability 
for service of the system. Such activities require the development of 
special recording, simulation, and processing tools; the development of 
thorough and highly coordinated test plans and procedures; the detailed 
analysis and study of system operation and of collected and processed 
operational data, Evaluat 3.0 n- - repeated- - can show the growth of service 
capability as a result of operator or user experience, fault correction, 
or system improvement. Human actions, reactions, and interactions 
determine, to a large degree, system service effectiveness and cost 
effectiveness. Evaluation serves to pinpoint procedures that can be 
improved and often indicates the nature of the solution. Often system 
tests can be used to support evaluation, 

A well designed system contains a recording function that collects 
data at a low recording level during operational periods to provide a 
record of performance and use. Increased recording, either preplanned, 
manually started, or automatically initiated as a result of a detected 
"condition”, should not interfere significantly with system operation yet 
should provide needed insight into system operation to assist in malfunction 
detection and correction. 



EXAMPLES 

Within a reasonably complex system one can expect to find many 
different types of documents. Each document type in the examples shown 
below serves a particular function. Although different "titles” could 
be used, and a different "packaging" approach could produce a different 
number of physical documents, the total information needs of the system 
are nearly invariant. 

Documentation must be viewed as serving specific needs, and the early 
detection and definition of such needs is a crucial stage of the system 
development process. Change control, maintenance, and updating of the 



; 



system itself must be performed in accordance with documented procedures 
and the impacted documents must be updated and modified in accordance 
with documented guidelines. The examples below , while not exhaustive , 
illustrate many of the system documentation needs. 



System Documents 

System documents include the: 



Modular Assembly Specification . One document for each identified 
module or subsystem that makes up the system. 

System Test Plan . Provides a schedule of system tests, locations, 
and times. “ 5 

.System Test Procedures . Defines tests and gives detailed informa- 
tion needed to conduct system tests. 

System Test Philosophy . Describes test approach, 

5 y s tern Des cr ip tion « A non— technical discussion and representation 
of system functions* uses, and relationships. 

End Item Documents 

These end item documents include: 

CPEI Specif iGatlon , Computer program end item specifications with 
one specification document for each program such as a monitor or read 
routine. Each specification document has two parts: 



Papt I of the specification contains the design requirements, 
design limitations, interface relationships, and critical 
parameter values. 

Part II of the specification contains the actual design 
information and operational details which adequately 
describe the produced product and its normal and peculiar 
operational characteristics and limitations, (Includes 
listings.) 



for eacn item such as a power supply, terminal, tape unit, or computer. 
Each specification (as described above for a CPEI) contains two parts. 



General System Specification , 
ign . 





A single document that provides Per- 
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Item lest Plan , An initial test plan document is prepared for each 
end item concurrently with the preparation of the Part I Specification. 

The final test plan is recorded when the final design of the end item is 
approved (before the item is built) , 

item Test Procedur e. An initial test procedure is prepared con- 
currently with the selection or development of the final design of the 
end item and the final procedure and required support tools are prepared 
by the time the item is built. 

Item Test Result . This document describes the results of applying 
the test procedures to the developed end item. This document is written 
afcer tests are performed in accordance with test procedures. 

Support Documents 

Documents in this category are identified (usually) with the 
development and operation of the actual system. These include t 

Procedural Manuals . Documents that describe and prescribe how func- 
tions or operations are to be performed. In CIS, most of these procedures 
will parallel traditional library activities; some, however, will provide 
guidance for new activities not previously performed by librarians (or 
users) , Many documents of this type are administrative, others are 
technical, and some are for training purposes. 

S tandards [ Manuals . Collections of guidelines used by system 
developers, operators, and users. Examples might include a glossary of 
terminology, a listing of official terms and parameters, programming 
conventions and restrictions, and charging (cost) algorithms. 

Operator Manuals . Documents developed for the guidance of those 
individuals that perform a role within the operational system (direct 
service, support, or maintenance). These documents present a detailed 
explanation of the interrelationships and features of each part of the 
system--and the environment in which the system operates. Each operating 
position is described in terms of the actions needed to support system 
performance . 

User Manuals . These documents are written to guide those who intend 
to use the system, but who do not need detailed knowledge of its inner 
workings. A user interfaces with the system but is not a "part" of the 
system although his presence may be needed before certain system functions 
become operable. This document provides the user with just enough detailed 
information so that he can interface effectively and satisfy his needs. 

The user manual also assists the user in interpreting system-generated 
output as well as serving as a guide to the preparation of user-generated 
input . 




be used to indoctrinate and train both users and operators * 

Plans, Schedules an d Reports 

Development schedules and mechanisms to report progress are essential 
S S^ ld f system development activities. Plans are needed to keep aspects 
or the development in perspective. These documents include* 

P.j-Bnning Doc uments ^ Documents that record intended actions and 

scheduled activities. They serve to inform and guide decision makers, and 
as such must be kept up-to-date. The planning document often contains 
time, manpower, and cost estimates related to defined tasks or goals. 

Development S chedule . A separate document or part of a planning 
document usually built around a milestone type presentation of time-phased 
tasks. Tasks are numbered, titled, and defined. Dependencies are in- 
cluded in milestone charts and critical paths are identified. Each task 
is identified with the person or group of organizational designation 
responsible for task completion during indicated time periods or by stated 
due dates. Some chart presentations do not indicate task dependencies, 

-Pj^QS^GSs Reports, These documents record actual progress in terms 
of established goals or tasks. The degree of detail can vary 
significantly. For a specific task, such as the preparation of a 
computer program or a user 1 s manual or a procedure T s manual, a method 
internal detailed reporting should be used that precisely indicates 
progress in terms of identified subtasks and expended manpower^ time, and 
cost. Reports that provide an overview of progress should not be as 
specific but should be based on detailed information. 



/ 
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IX, SYSTEM TEST AND EVALUATION CONSIDERATIONS 



During 
that users T 
information 
system test 
developed , 
cedures can 
required in 
required in 



Phase IIB the primary concerns of the Institute are to insure 
information needs and users 1 methods of manipulating such 
are defined in terms of the CIS program, and that adequate 
and system evaluation concepts, plans and schedules are 
These products are needed so that the proper tools and pro- 
be prepared to support both the operational validation tasks 
Phases III and IV and the system capability investigations also 
Phases III and IV and during ongoing operation. 



GOALS 

It seems self evident that user satisfaction is the ultimate factor 
in evaluating a service system. While there are many subfactors involved 
in user satisfaction, the fact remains if the users do not like the 
service it must be considered a failure regardless of how perfect the 
system may be in terms of cost, equipment, programs and efficiency. 
Therefore, an important goal for this phase of the CIS project is to 
determine the nature of the information needs of potential users and how 
they would employ the data derived from CIS. One approach to this problem 
is through the established campus User Committees ; another is through user 
surveys . 

A second goal for this phase of the project is to develop methods, 
concepts, plans and schedules for testing and evaluating the system in 
light of the needs and services required by CIS users. Many factors and 
issues must be considered in developing the various test and evaluation 
procedures. Both of the above goals must be realized in order to carry 
out the operational validation work required in the later phases of the 
project. 



SYSTEM TESTING 



Operational validation, more properly called system testing, is 
performed under controlled conditions and for specified periods to 
determine that the performance of the CIS system (computer programs, 
hardware, humans, and procedures) meets the performance and design 
requirements contained in the general system specification. System 
tests are designed to require perfect human performance since the goal 
is to systematically test all combinations of software products and 
hardware elements in an operational setting. Test results are 
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established using specific data recorded during test operations. Pre- 
specified measures are computed and the values compared with pre-established 
limits of acceptable performance (for stated confidence levels) . These 
limits or limiting values may be included in the system specification; more 
often, the actual values are determined during subsystem testing or during 
an operational shakedown period scheduled as part of the system implemen- 
tation phase. System tests are preceded by other testing phases (item, 
module , and subsystem) , 

Operational Shakedown 

In general, the operational shakedown period is one in which both the 
people who developed the system and the people responsible for ultimate 
hands-on operation of the system work together to test system procedures 
and working relationships. This period is also used to test system 
maintenance tools and techniques; to establish acceptable performance 
norms; to detect and correct system errors and gross system weaknesses; 
and to give the operating personnel confidence in the system and in their 
ability to operate the system successfully. As mentioned elsewhere, 
system def iciencies--previously unsuspected--are often detected during 
shakedown. These new insights give rise to planned improvement activities; 
to the development of new simulation tools, processing programs, and 
analysis techniques; to various research programs or studies directed 
toward areas of uncertain or poor knowledge; perhaps even to the awareness 
of new concepts of operation or service, 

SYSTEM EVALUATION 

System evaluation is directed toward testing an operating system in 
an uncontrolled (but measured) operational environment. The goals are 
to determine how well the system performs under a wide range of operating 
conditions; to determine actual capability levels inherent in the system 
design; and to detect problem areas, subtle as well as major. Testing is 
in-depth, pre-planned, and especially sensitive to imperfect human 
performance. The latter may be caused by improper personnel assignments, 
inadequate training, incorrectly designed man-machine interfaces, personnel 
problems, or other causes. The intention of planned evaluation tests is 
to gather data needed to compute performance measures similar to those used 
for system test, to establish the validity of on-going operational measures 
of performance, and in each case to father all of the supporting data 
needed to conduct in-depth system analysis, 

, i 

The previous reference to an "Uncontrolled operational environment" 
is interpreted to mean 'o artificial operational constraints are introduced 
into the system (or po ion of the system) being tested and that the 
testing activity, inel ing special data recording and measurement, does 
not destroy operationa validity. It is true, however, that a wide range 
of possible operation' , loads and environmental perturbations are 
deliberately introduced so as to exercise the operational functions under 

O 
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test. In some instances, simulation techniques are used to create the 
desired load situation. The same simulation devices also serve as 
research tools to support system design as well as system evaluation 
efforts. Often the end product of evaluation is a redesign of some 
system function or method of implementation. 

In a system such as CIS, the user becomes a prime agent in the 
testing and evaluation process. Many information handling processes 
can be examined as to cost of implementation, cost of operation, or 
cost of services rendered; however, the value or benefit of a particular 
parameter (such as response time) is exceedingly hard to determine. 
Benefit-value, or benefit-cost as it has been called, is affected by so 
many variables that it usually eludes definition as well as measurement. 
Such insight is often gained too late for initial decision-making 
purposes; however, the anticipated qualitative value of an activity that 
advances the state of the art must always be kept in mind to help balance 
the more quantitative, often short range, values derived from cost-benefit 
analysis , 



LIBRARY EFFECTIVENESS AND CRITERIA 

A recent report* published by the Institute contains the results 
of a literature search to locate measures of library effectiveness. 

The investigators established six basic concept criteria of library 
performance as shown on Figure 11. In this context a criterion concept 
is the abstract variable which needs to be measured or evaluated, e.g,, 
cost, use, response time, etc. A criterion measure is the operational 
definition or method by which the criterion will be quantified. Using 
previous Institute work in this area as a basis the project can de- 
termine which concepts and measures are appropriate for use in terms of 
the CIS program. The six concepts are*. accessibility, cost, use, user 
satisfaction, response time, and cost-benefit. Ability of the system to 
provide service, and adequacy of the available data resources, determine 
the degree to which the system is used and of user satisfaction. Taken 
together, these issues establish the total economic profile which, in the 
final analysis, is an issue of constant, often grave concern to both user 
and library. 

Knowledge of the characteristics of the users--and of the searchers-- 
can help define system requirements, both initially and for future 
improvements. Methods of service, with variations, range from batch 
processing to on-line controlled computer searches in real time. System 
reaction depends on whether the system controls the search formulation, 
whether the searcher can force the system to accept (and interpret) his 



Evans, G. Edward and Harold Berko, Effectiveness Criteria for 

Medical Libraries , Institute of Library Research, University 
of California, Los Angeles, April 1970, 

0 



FIGURE 11 



CONCEPTS AND MEASURES OF LIBRARY EFFECTIVENESS 



1, ACCESSIBILITY 

Number of services and applicability to classes of users. 

Ratio of services requested to services available. 

Ratio of holdings to total user population (actual, potential) » 

2, COSTS 



Staff size. 

Staff skill and characteristics. 
Unir cost. 

Ratio of book budget to users . 



3 . USE 



Gross use of services. 

Ratio of actual users to potential users. 

Total library use. 

Ratio of service use to total number of users. 

Ratio of total use to total number of services. 

Percentage /of materials used (by type and by class of user) , 
Ratio of documents circulated to various classes of users. 
Ratio of documents circulated to active users. 

Ratio of total use to total holdings. 

Item-use-day . 



4, USER SATISFACTION 




User satisfaction. 

User activities (purpose) in library. 

Percentage of items in collection as listed in a checklist. 
Percentage of items in collection by type of material. 



Percentage of various item types to classes of users. 
Quality-value of collection items based on expert opinion, 
Ratio of documents used /to materials requested. 



I: 
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5 . RESPONSE TIME 

Speed of service. 

Ratio of number 1 of services offered to average response time. 
Ratio of response tin, 3 to total time document is of value. 

Ratio of holdings to response time, 

6, COST-BENEFIT 

Ratio of services to cost. 

Ratio of total service expenditures to users (actual, potential) , 
Cost-benefit, 

Cost-response time. 



P 
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f nl I statement of need; or whether the search formulation is a shared 
responsibility with limitations on both sides. Library effectiveness 
studies and operational effectiveness approaches as applied to computer- 
based information handling systems serve as resource material to develop 
specific system test and system evaluation measures. In addition, opera 
tional effectiveness determination as an ongoing activity (to provide a 
day by day profile of system performance) also must be included in CIS 
design requirements. 



OPERATIONAL EVALUATION AND FEEDBACK 

I 

The purpose of the CIS project is to provide information services. 
To this end, user satisfaction is a primary indication of the success 
of the project. Any amount of system efficiency and sophistication will 
be insignificant if the users are not receiving needed services. 

User Satisfaction 



A fairly comprehensive, but hopefully not overly expensive, method 
of evaluating user satisfaction must be designed in order to determine 
needed system improvements during both preliminary, experimental use of 
the system and during later production runs. With bibliographic service? 
initially emphasized more than the textual and numerical services, more 
attention is given here to user feedback concerning the bibliographic 
services. It is expected that users interested primarily in textual and 
numerical services will form a smaller, more sophisticated user community 
that will be able to involve itself to some extent in the development of 
service specifications. 

The major questions central to user service evaluation are: 

1. What measurements do we wish to have? 

2. What are the best techniques we can use to get these 
measurements? 

3. What system characteristics do we wish to evaluate with 
our measuring techniques? 

Traditionally, two parameters have been used as measurements of user 
satisfaction with SDI and retrospective searching: _ recall (the ratio of 
hits received to the total number of hits in the file) and precision (the 
ratio of good hits to the total number of hits) . Common sense seems to 
verify the importance of these parameters. However, ether factors such 
as user characteristics, response time, cost, and output format 
acceptability must also be taken into consideration. 

Among the techniques used to receive feedback from users of 
bibliographic services are personal interviews, questionnaires and 



response cards, parallel or sample runs, studies of hits and pertinency, 
the personal experience of the information specialist who interfaces the 
user with the system, and the analysis of operational use parameters 
recorded for this purpose. The amount of usage of the CIS system (and 
how much users are willing to pay) should also reflect to some extent 
the degree of user satisfaction. The advantages, disadvantages, and cost 
of such techniques for data gathering must be weighed and a set of pro- 
cedures (perhaps combining the techniques) must be developed and used. 

Closely related to both the measurement of user satisfaction and the 
techniques of measuring are the particular system characteristics to be 
evaluated. These characteristics should (at the time of evaluation) be 
changeable. Examples of these characteristics are file inversion, depth 
of subject indexing, profile complexity, searching on terms in the title 
vs. searching on terms in the abstract vs. searching on assigned de- 
scriptors, and the amount of user interaction at request time. 

/ 

Operational Performance Considerations 

In order to give users the best service possible, it is important to 
maintain a well- tuned, efficient system and to be able to adjust the 
system from time to time so as to obtain the maximum computing power. 

This means that measurements must be collected by the software in order 
to pinpoint areas that are costly, time or space-consuming, or, in genera 
are operating inefficiently. Four types of measurements that will be 
collected include software operating statistics, comparisons with other 
systems, turn around time, and system failures. 

Software Operating Statistics . Built into the CIS System will be a 
Statistical Log on which any component of the system may write operating 
statistics. Such statistics could include CPU time, real time, I/O 
requests, number of profiles passed, number of records searched, and 
similar measures used to pinpoint bottlenecks in the system, to determine 
which 7 routines could better be written in the assembler language for 
better optimization, and to discover other ways to rewrite parts of the 
system to produce an efficient production system. Such a log also will 
assist, in determining the proper charging algorithms. System capabilities 
should permit the detailed measurements to be suppressed when only 
operational recording is required. 

C omparison with Other Systems . Simultaneously with the development 
of the CIS system, experimental data bases services will be provided using 
a modified IBM TEXT-PAC system and the modular retrieval program described 
in Part 1. Results can lead to both understanding and improvement of the 
CIS programs. These comparisons will be, initially, at a gross level of 
measurement. 

Turn Around Time . Not only is it important that the throughput of 
the software system be monitored, but it is equally important that the 
rest of the system be monitored as well. In this area the concern will 
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be with measurements such as: how long it takes to get a new profile 

prepared, how long does it take to get a new tape ready for use by the 
programs, hoxv long does it take to get the output returned to users, etc,? 
(Note: if the system is interactive, many of these measurements can be 

taken by the software; otherwise people have to record them.) 

System Failures . Performance data should indicate how often the 
system fails due to such things as software bugs, hardware and operating 
system failure, clerical errors made during preparation of profiles and 
data bases, etc. Another kind of "failure" that will be interesting to 
monitor is the following: how often do we have to say "no" to a user’s 

request because of lack of capability in our system, or a bug that has 
not been corrected? This kind of measurement will provide us with 
information concerning much-needed new or improved services. 



Accounting 

Customer charging algorithms will be needed. These will be based on 
parameters which are somewhat controllable by the user and should, within 
reason, be both predictable iid repeatable. At the same time, they must 
provide the necessary cost recovery for the service organization. Since 
one goal of the CIS project is to reduce processing costs through resource 
sharing according to the available set of requesters, actual computer 
charges alone will not serve as they cannot be predicted or repeated. In 
order to recover costs through prices based on user-controllable parameters, 
it probably will be necessary to use statistical techniques and to weight 
the selected charging components according to anticipated system usage 
and resource sharing. Other options will be considered. 

Gathered statistics will provide the information necessary to project 
estimates used in pricing, and the program will later provide the trans- 
action reports used to generate the bills themselves. In general, an 
executing service will log out charge-related statistics for each requestor 
for which individually-accountable work was done, plus similar statistics 
for any collectively-accountable work, along with a list of the requester 
set. These statistics might include: 

1. The number of entries passed for each file used, 

2. Elapsed real time, 

3. Processor time and billable I/O activity, as defined by the 
host operating system, 

4. Main storage used. 

From such statistics, a charging scheme can be derived by steps such as: 

1. Listing user-controllable resources which are 
subjectively acceptable billing parameters. 
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2. Obtaining relations between total cost of operation over 
a sample period and the total amount of each reported 
resource used. 

3. Building a formula, linear in the significant parameters, 
which tends to correlate highly with actual costs of 
operation. 

4. Weighting the values of the formula to produce the 
required dollar value based on the observed total cost 
of operation, plus appropriate percentages of personnel 
costs, etc. 

All library functions that are impacted by CIS will be carefully moni- 
tored so that adequate records of personnel utilization, incurred costs, 
and time expenditures are available for analysis. The degree to which 
these costs are to be reflected in customer charges will be an area of 
study and consideration throughout the CIS project. Recording of resources 
usage and recording of problem symptoms will take place in the library as 
well as in the computing center, and may come from allied information 
specialists as well. 




APPENDIX A 

PRELIMINARY PHASE IIB DEVELOPMENT PLANNING INPUTS 
PROPOSED DEVELOPMENT SCHEDULE 



The ten major task areas initially planned for Phase IIB (assuming 
a 12 month period) are shown on Figure A-l. The responsible development 
group (CCN, Library, or ILR) is indicated for each area, with two groups 
sharing responsibility for Area 4 (Experimental Services) and Area 7 
(User Community Interface) . The associated time schedule* indicates a 
reasonable task phasing. The material in this appendix was developed as 
resource material used to prepare the Phase IIB proposal. 



Task Areas 

The task areas shown on Figure A-l are described below: 

1. Experimental Operations . The conduct of experimental opera- 
tions with interim software will provide valuable information 
concerning real-time operational problems and procedures, 
evaluation and monitoring techniques, user interface problems, 
and the quantity and quality of services needed. This 
information can serve as input to the design of the CIS 
software and to the design of the system as a whole. Most 
of the experimental operations will center around the use of 
the TEXT-PAC system. Plans call for providing SDI searches 
of the CA Condensates File which is produced by the Chemical 
Abstracts Service (CAS) . A modest amount of effort will be 
needed to modify the TEXT-PAC programs to accept CAS T s new 
Standard Distribution Format (SDF) ; however, initial operations 
will begin using the old format. Depending upon the acquisitions 
schedule being prepared by the Library, other data bases may be 
processed by TEXT-PAC during the year. Other assets include 
computer programs such as the ones developed to provide 
searching, indexing, and processing of the ERIC File 
(see Part 1 of this Final Report) and others that provide 
on-line text formatting and manipulation capabilities. 



*The current Phase IIB effort is based or. an 18 month development 
period funded at a lower level than used to develop the schedule 
in this appendix. 
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PHASE I IB - CIS DEVELOPMENT SCHEDULE 
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2. Prototype Development . When the software design has progressed 
sufficiently, construction of a prototype of the system will 
begin. The prototype will serve primarily to test certain 
design decisions and will not necessarily represent the entire 
system. The initial or interim version of the prototype will 
probably contain some of the bibliographic services. The 
degree of progress is dependent on the feedback obtained from 
experimental operations, user interface studies, and library 
inputs . 

3. System, Design and Documentation ,., A set of specifications for 
the CIS software system will be produced along with a descrip- 
tion of the logical components of the system. At regular 
intervals during the design stage, these documents will be 
updated to reflect the current view of the system. A final 
set of documents will be produced for use in Phase III. 
Considerable attention will be given to the preparation of 

a development plan and schedule for Phase III to insure 
continuity of design development and continuation of the 
services already being offered to users. Development status 
will be reflected in Quarterly Progress Reports. 

4. Experimental Services . Both CCN and the Library will work 
together to provide a set of experimental services, with 
particular emphasis on identifying user groups and user 
needs, and on acquiring user feedback and guidance. Library 
personnel will be involved. 

5. Experimental Acquisition . One of the major functions per- 
formed by the Library during Phase IIB will be to acquire 
needed available data bases which can be used to support 
experimental services. 

6. Procedures Development . Successful CIS operation requires 
a set of library procedures for data base selection, 
acquisition, cataloging, and announcement. These procedures, 
including guidance documents for information specialists, 
will be prepared by library personnel under Institute 
guidance . 

7. User Community Interface . Both the Library and ILR will 
work closely with the user community (particularly with the 
Users Committees and the Extra-Campus Committee) to determine 
user needs, identify and inventory available data bases, 
select for acquisition, and obtain user feedback. Resources 
of other University of California campuses will be surveyed 
through interface relationships with the University of 
California Library Systems Development project and the CIS 
Extra-Campus Committee. 
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8. CIS Seminars . Current plans call for one or two seminars to be 
presented during Phase IIB; however, the format may be modified 
to reflect more advanced CIS development needs. 

9. Data Base Experimentation . Efforts aimed at develooing improved 
data manipulation and search techniques, such as superimposed 
coding, will be pursued during this phase. 

10. Test and Evaluation Studies . Serious studies will be conducted 
to determine an appropriate approach to testing the CIS system 
during implementation and to evaluating the operational system. 
Based on studies of user needs and modified ’ y user feedback, 
measures of library effectiveness and user satisfaction will 
be investigated for both test and evaluation purposes. 



Documentation Topics 

The time schedule shown on Figure A~1 will be updated at tt- be- 
ginning of the first and third quarters of 1971 as shown by Topic K 
(Update Phase IIB Development Schedule) on Figure A-2. The entries on 
this latter figure show the task oriented documentation topics identified 
for Phase IIB. Most of these outputs are required for Phase III develop- 
ment activities. Documentation contributions and responsibilities are 
tentatively indicated for each development group, with the responsible 
group underlined . In most cases , the documents are prepared for use or 
release by the Institute of Library Research. In several instances, the 
review and approval of each group is needed before the document can be 
released. Basic development and resource allocation decisions require 
approval of the Principal Investigator. 

The documentation topic outline is presented as a task development 
guide. While it does not necessarily identify individual documents, it 
does indicate that written material will be generated as required in the 
identified development areas to provide needed inputs fur Phase III. 

Some of the material may appear twice during the year, with the second 
issue being used to revise or augment the first issue. Man}' of the 
documented topics will be updated, rewritten, or completed in Phase III 
to reflect development and operational progress. During Phase IIB, most 
of the technical documents will be developed and used as working drafts 
with no attempt being made to publish the document in final form until 
the content is complete and accurate. 




- 60-65 



PHASE I IB - DOCHMKNTATION TOPICS 
(Based on 12 Month Period) 
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